Assessing the efficiency of Laser-Induced Breakdown Spectroscopy (LIBS) based sorting of post-consumer aluminium scrap

The aluminium Twitch fraction of a Belgian recycling facility could be further sorted by implementing Laser-Induced Breakdown Spectroscopy (LIBS). To achieve this goal, the presented research identifies commercially interesting output fractions and investigates machine learning methods to classify the post-consumer aluminium scrap samples based on the spectral data collected by the LIBS sensor for 834 aluminium scrap pieces. The classification performance is assessed with X-Ray Fluorescence (XRF) reference measurements of the investigated aluminium samples, and expressed in terms of accuracy, precision, recall, and f1 score. Finally, the influence of misclassifications on the composition of the desired output fractions is evaluated.Peer ReviewedPostprint (published version

Enhanced plastic recycling using RGB+depth fusion with massFaster and massMask R-CNN

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe rapid increase in waste generation from electrical and electronic equipment (WEEE) has created the need for more advanced sensor-based systems to sort this complex type of waste. Therefore, this study proposes a method for object detection, instance segmentation, and mass estimation of plastics and contaminants using the fusion of RGB and depth (D) images. The methodology is based on the Faster and Mask R-CNN with an extra head for the mass estimation. In addition, a pre-processing method to enhance the depth image (ED) is proposed. To evaluate the data fusion and pre-processing method, two data sets of plastics and impurities were created containing images with and without overlapping samples. The first data set contains 174 RGB images and depth (D) maps of 3146 samples, excluding their mass value, while the second data set contains 42 RGB and D images of 766 pieces together with their mass. The first and second data sets were used to evaluate the performance of Mask and Faster R-CNN. Further, the second data set was used to evaluate the network’s performance with the additional head for mass estimation.The proposed method achieved 0.75 R 2 , 1.39 RMSE, and 0.81 MAE with an IoU greater than 50% using the network Resnet50_FPN_RGBED. Hence, it can be concluded that the presented method can distinguish plastics from other materials with reasonable accuracy. Furthermore, the mass of each detected particle can be estimated individually, which is of great relevance for the recycling sector. Knowing the mass distribution and the percentage of contaminants in a waste stream of mixed plastics can be valuable for adjusting the parameters of upstream and downstream sorting processes.Peer ReviewedPostprint (author's final draft

Simultaneous mass estimation and class classification of scrap metals using deep learning

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksWhile deep learning has helped improve the performance of classification, object detection, and segmentation in recycling, its potential for mass prediction has not yet been explored. Therefore, this study proposes a system for mass prediction with and without feature extraction and selection, including principal component analysis (PCA). These feature extraction methods are evaluated on a combined Cast (C), Wrought (W) and Stainless Steel (SS) image dataset using state-of-the-art machine learning and deep learning algorithms for mass prediction. After that, the best mass prediction framework is combined with a DenseNet classifier, resulting in multiple outputs that perform both object classification and object mass prediction. The proposed architecture consists of a DenseNet neural network for classification and a backpropagation neural network (BPNN) for mass prediction, which uses up to 24 features extracted from depth images. The proposed method obtained 0.82 R2, 0.2 RMSE, and 0.28 MAE for the regression for mass prediction with a classification performance of 95% for the C&W test dataset using the DenseNet+BPNN+PCA model. The DenseNet+BPNN+None model without the selected feature (None) used for the CW&SS test data had a lower performance for both classification of 80% and the regression (0.71 R2, 0.31 RMSE, and 0.32 MAE). The presented method has the potential to improve the monitoring of the mass composition of waste streams and to optimize robotic and pneumatic sorting systems by providing a better understanding of the physical properties of the objects being sorted.Peer ReviewedPostprint (author's final draft

Detection and recognition of batteries on X-Ray images of waste electrical and electronic equipment using deep learning

The trend of increased use of lithium-ion batteries, challenges the cost-effectiveness and safety of manual battery separation during the end-of-life treatment of Waste Electric and Electronic Equipment (WEEE). Therefore, the need for novel techniques to separate and sort batteries from WEEE is increasingly important. For this reason, the presented research investigates the potential to facilitate the development of novel techniques for battery extraction and sorting by examining the technical feasibility of predicting the presence, location, and type of batteries inside electronic devices with a deep learning object detection network using X-Ray images of the internal structure of WEEE. To determine the required X-ray imaging parameters, 532 electronic devices were arbitrarily collected from a recycling facility. From each product, two X-Ray Transmission (XRT) images were captured at two different X-Ray source configurations. Results obtained with the limited dataset are promising, demonstrating a 91% true positive rate and only a 6% false positive rate for classifying battery-containing devices. Moreover, a precision of 89% and a recall of 81% are demonstrated for battery detection, and an average precision of 85% and an average recall of 76% are demonstrated to distinguish amongst the following six battery technologies: cylindrical nickel-metal hydride or nickel-cadmium, cylindrical alkaline, cylindrical zinc-carbon, cylindrical lithium-ion, pouch lithium-ion, and button cell batteries. These results demonstrate the potential of using deep learning object detection on XRT-generated images for both automated battery extraction and sorting, regardless of the condition or shape of the products.status: publishe

You Only Demanufacture Once (YODO): WEEE retrieval using unsupervised learning

Recent developments in robotic demanufacturing raise the potential to increase the cost-efficiency of recycling and recovering resources from Waste of Electrical and Electronic Equipment (WEEE). However, the industrial adoption of robotic demanufacturing for mixed WEEE streams requires tailored instructions for every product model. Considering the large variation in product models, it is not expected to be feasible in the coming decade to rely only on computer vision technologies to define the tailored instructions required for robust and time-efficient robotic demanufacturing. Therefore, the presented research developed a generic retrieval system named You Only Demanufacture Once (YODO) based on content-based image retrieval (CBIR) to identify the product model and retrieve product model-specific demanufacturing instructions. The system compares the visual features represented on a color image of the WEEE with a database of known descriptions representing previously imaged WEEE to find a match or to figure out whether the analyzed product model is new to the system. The performance of YODO is evaluated with a case study for laptop model identification, where a large dataset is created including 4089 images of a representative laptop waste stream. The results demonstrate a top-1 retrieval mean average precision (mAP) of 93.75%. After running YODO on 3600 laptops, the system learned 1079 unique product models, and the presented results show an 85% chance that the next laptop presented to the system is already registered in the database, allowing the retrieval of relevant information for robotic demanufacturing. This corroborates that a fast learning rate can be achieved, allowing a YODO system to support the robotic demanufacturing by making prior product-specific learnings available.Award-winningPostprint (author's final draft

Deep learning regression for quantitative LIBS analysis

One of the most promising innovation strategies for sorting and recycling post-consumer aluminium scrap is using quantitative Laser-Induced Breakdown Spectroscopy (LIBS) analysis. However, existing methods to estimate alloying element concentrations based on LIBS spectra, such as linear univariate regression and Machine Learning models, are still too limited in their performance to achieve the accuracy demanded by the industry. Therefore, this study presents novel Deep Learning approaches and compares their performance to those of traditional univariate regression and Machine Learning methods in terms of RMSE, MAE, and R2 value. For this evaluation, two sample sets of aluminium pieces are used: one containing 27 certified aluminium reference samples and the second containing 733 post-consumer scrap pieces for which the ground truth concentrations are determined by X-Ray Fluorescence (XRF). Adopting multiple loss functions, one for each element, has proven its significant value for the regression performance. It improves the results for all performance metrics in the Scrap Sample set, and the same is true for the Reference Sample set, except for the coefficient of determination of Fe, Mn and Mg. In addition, the proposed methodology considers the learning prioritisation problem to prevent that learning the concentration of the base element is prioritised over the alloying elements. Although the effect of excluding the base alloy aluminium from the learning is small and not always positive for the performance, demonstrating this effect is also considered valuable. Since the average RMSE on the prediction is just 0.02 wt% for Al and Si, and not more than 0.01 wt% for Fe, Cu, Mn, Mg, and Zn, the best-performing Deep Learning model shows promise for the future of LIBS in metal sorting applications.Peer ReviewedPostprint (author's final draft

Real-time classification of aluminum metal scrap with laser-induced breakdown spectroscopy using deep and other machine learning approaches

In the recycling industry, the use of deep spectral convolutional networks for the purpose of material classification and composition estimation is still limited, despite the great opportunities of these techniques. In this study, the use of Laser-Induced Breakdown Spectroscopy (LIBS), Machine Learning (ML), and Deep Learning (DL) for the three-way sorting of Aluminum (Al) is proposed. Two sample sets of Al scrap are used: one containing 733 pieces for pre-training and validation with a ground truth of X-Ray Fluorescence (XRF), and the second containing 210 pieces for testing for unknown compositions. The proposed method comprises a denoising system combined with a method that extracts 145 features from the raw LIBS spectra. Further, three ML algorithms are assessed to identify the best-performing one to classify unknown pieces of aluminum post-consumer scrap into three commercially interesting output classes. The classified pieces are weighed, melted, and analyzed using spark analysis. Finally, to optimize the best-performing ML system, three state-of-the-art denoising and three feature extraction networks are pre-trained for learning the baseline correction and the proposed feature extraction. Transfer Learning from the six pre-trained networks is applied to create and evaluate 24 end-to-end DL models to classify Al in real-time from >200 spectra simultaneously. The end-to-end DL scheme shows the advantages of learning and denoising the spectra, allowing the transfer of traditional spectral analysis knowledge and the proposed feature extraction into DL, where the network learns from the entire spectrum. The best results for ML and DL were obtained with Random Forest processing one spectrum in 150 ms and BPNN+GHOSTNET(Fine-tuning) processing 200 spectra in 9 ms, which achieved 0.80 Precision, 0.81 Recall, 0.80 F1-score, and 0.80 Precision, 0.79 Recall, 0.79 F1-score, respectively.Peer ReviewedPostprint (author's final draft

Classification of aluminum scrap by laser induced breakdown spectroscopy (LIBS) and RGB + D image fusion using deep learning approaches

Integrating multi-sensor systems to sort and monitor complex waste streams is one of the most recent innovations in the recycling industry. The complementary strengths of Laser-Induced Breakdown Spectroscopy (LIBS) and computer vision systems offer a novel multi-sensor solution for the complex task of sorting aluminum (Al) post-consumer scrap into alloy groups. This study presents two novel methods for fusing RGB and Depth images with LIBS using Deep Learning models. The first method is a single-output model that combines LIBS UNET and two DenseNets in a late fusion framework. The second method is a multiple-output model that uses the structure of the single-output model to enhance learning and avoid overfitting. In particular, the network has two outputs that enable the regularization of the individual sensors. A data set of 773 aluminum scrap pieces was created with two sets of ground truth-values, corresponding to the two envisaged sorting tasks, to train and evaluate the developed models. The first sorting task is separating Cast and Wrought (C&W) aluminum. The second is the division of the post-consumer aluminum scrap into three commercially interesting fractions. The single-output model performs best for separating C&W, with a Precision, Recall, and F1-score of 99%. The multiple-output model performs best for classifying the three selected commercial fractions, with a Precision, Recall, and F-score of 86%, 83%, and 84%, respectively. The presented data fusion method for LIBS and computer vision images encompasses the great potential for sorting post-consumer aluminum scrap. By sorting mixed post-consumer aluminum scrap in alloy groups, more wrought-to-wrought recycling can occur, and quality losses can be mitigated during recycling.Peer ReviewedPostprint (author's final draft

Forecasting global aluminium flows to demonstrate the need for improved sorting and recycling methods

The probable emergence of a global aluminium scrap surplus in the coming decade is one of the main incentives for the aluminium recycling industry to invest in new methods and technologies to collect, sort and recycle aluminium scrap. However, due to the considerable uncertainty in the evolution of the global scrap surplus, it is difficult for policymakers and the recycling industry to accurately estimate the economic and environmental advantages of implementing enhanced sorting and recycling methods. The International Aluminium Institute (IAI) has developed a model to track and forecast the global flows of aluminium, but this model is not extensive enough to estimate the scrap surplus evolution. Therefore, this paper introduces an alloy series resolution to the supply and demand of aluminium in the IAI’s global flow model and estimates the composition of the recovered scrap flows to improve the estimate of the technical potential of secondary alloy production. The estimated scrap surplus evolution is subjected to a sensitivity analysis, considering the most critical parameters, including the speed of electrification in the automotive sector, the recovered scrap’s composition and the lifetime of aluminium products. In addition, the estimated composition of the recovered aluminium scrap in the model is compared to composition measurements of alumimium scrap collected at a Belgian recycling facility as a means of validation. This study allows to estimate that the global aluminium scrap surplus will emerge soon and reach a size of 5.4 million tonnes by 2030 and 8.7 million tonnes by 2040, if currently adopted aluminium sorting and recycling methods are not improved.Peer ReviewedPostprint (author's final draft

Quantification of alloying elements in steel targets: The LIBS 2022 regression contest

We present the results of the regression contest organized for the LIBS 2022 conference. While the motivation and design of the contest are briefly presented, the work focuses on the methodologies of the three best-performing teams. The employed spectral preprocessing strategies, choice of regression models and its optimization are detailed for each team separately. The aim of the contest reflects the long-term challenges faced by quantitative laser-induced breakdown spectroscopy (LIBS) analysis. Thus, the contest was designed with the purpose of providing a transparent platform for comparing and evaluating the large range of data processing tools available in the LIBS literature. Namely, the contest consisted of the quantification of two major (Cr, Ni) and two minor (Mn, Mo) elements in 15 steel targets. For constructing an appropriate regression model, spectra of 42 targets were provided. The spectra were collected using a commercially available laboratory-based LIBS system and made publicly available. The contest lasted 53 days during which the teams did not receive feedback. In total, 21 teams participated out of which the three best-performing methodologies are presented here. A single linear partial least squares model and two artificial neural network regression models are presented. The corresponding feature selection strategies included emission line selection, spectral range selection, and automatized wavelength selection. Various spectral normalization strategies and data augmentation strategies are also presented.JV acknowledges the financial support provided through the grant TACR TREND 6 - FW06010042 (Research and development of an advanced interaction vacuum system for laser spectroscopy). PP and EK acknowledge the financial support provided through the grant NCK II - TN02000020 (CAEPO: Center for advanced electronics and photon optics).Peer ReviewedPostprint (author's final draft